Reply to "Interlaboratory and interstudy reproducibility of a novel lateral-flow device: a statistical issue".

Iwas interested to read the paper by N. P. Wiederhold and colleagues published in the November 2012 issue of the Journal of Clinical Microbiology (1). The authors aimed to determine interlaboratory and interstudy reproducibility as well as to assess the effect of antifungal therapy on a lateral-flow device (LFD) developed for invasive pulmonary aspergillosis (IPA) detection. They reported a good interlaboratory agreement (LFD result agreement, 97% [32/33] of the serum and 78.8% [26/33] of the BAL samples from infected animals) and a good interstudy agreement (1). Such a descriptive result has nothing to do with reliability and actually is one of the common mistakes in reliability analysis (2–6). Moreover, Wiederhold et al. reported that the serum sensitivity of each surrogate marker assay was reduced in animals treated with antifungals. In contrast, the markers remained elevated within the BAL fluids of treated animals, demonstrating that the results of the LFD assay are reproducible between different laboratories and studies (1). Sensitivity is one of the seven well-known statistical estimates for validity (diagnostic accuracy) analysis and should not be used for reliability purposes (2–6). Why did the authors not use well-known intraclass correlation coefficient (ICC) or weighted kappa values to assess reliability (2–6)? Reliability and validity are two completely different methodological issues in research. Sensitivity, specificity, positive predictive values, negative predictive values, likelihood ratios of positive results (true positive/false positive), and likelihood ratios of negative results (false negative/true negative) as well as odds ratios (true results false results, preferably more than 50) and diagnostic accuracy are among the tests to evaluate the validity (accuracy) of a single test compared with a gold standard (5, 6). As a take-home message, for reliability and validity, appropriate statistical tests should be applied.